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OBJECTIVE-  The  research  is  investigating  the  use  of  Virtual  Reality  Environments  (VE)  to  visualize 
complex  data  and  information.  Virtual  Reality  (VR)  combines  3D  views  with  animation  to  allow  the  user  to 
navigate  through  the  visualization.  This  application  differs  greatly  from  representing  real  worlds  in  VR  for 
say  training  or  demonstration.  A  generic  display  model  for  visualizing  information  in  a  VR  is  being 
developed.  This  model  will  extend  previous  research  on  2D  and  3D  models.  Information  visualization  is 
vital  to  the  day-to-day  operations  of  Navy  personnel  planning,  distribution,  and  assignment. 

The  goal  of  this  research  is  the  construction  of  a  useful  and  generic  information  display  model  for  use  in 
advanced  visualization  environments.  The  display  model  will  consist  of  a  number  of  operators  that  act  on 
data  to  transform  it  into  visual  representations  in  virtual  reality.  The  operators  will  take  advantage  of 
human  perception  features  not  allowed  by  other  mediums.  These  features  include  direction  of  motion, 
stereoscopic  depth,  and  lighting  directions.  This  research  addresses  the  future  directions  of  human 
computer  interaction  and  interfaces,  and  information  visualization. 

APPROACH:  An  experimental  software  platform  is  being  developed  to  test  and  evaluate  the  usability  and 
suitability  of  the  display  model.  The  system  will  be  developed  to  utilize  advanced  technology  in  virtual 
reality.  A  virtual  reality  development  language  will  be  utilized  that  is  compatible  with  VRML  and  the 
CAVE  virtual  environment.  The  application  here  is  to  the  visualization  of  large-scale  software  systems.  A 
prototype  software  system  is  being  developed  as  an  experimental  test  bed  for  experimentation  of  the 
developed  methods. 

CONCLUSIONS :  No  visualization  method  addresses  all  the  needs  of  the  users.  One  successful  approach 
to  address  more  of  the  user’s  needs  is  to  offer  multiple  views  of  the  data  as  done  by  [Kmght'Ol,  Reiss’01, 
Storey'01].  Using  one  view  of  the  data  limits  the  number  of  attributes  and  the  available  exploration  space. 
The  solution  we  propose  to  overcome  this  problem  is  the  efficient  use  of  a  3D  space  for  visualization. 

Visualization  in  the  2D  space  has  been  actively  explored.  Many  techniques  for  generating  diagrams, 
graphs  and  mapping  information  to  the  2D  representation  have  also  been  studied  extensively.  Although  the 
question  of  what  benefits  3D  representation  offer  over  2D  still  remains  to  be  answered,  some  experiments 
have  given  optimistic  results.  These  results  further  motivate  our  work  presented  here. 

The  work  of  Hubona,  Shirah  and  Fout  [Hubona'97]  suggest  that  users'  understanding  of  a  3D  structure 
improves  when  they  can  manipulate  the  structure.  Ware  and  Franck  [Ware'94]  indicate  that  displaying  data 
in  three  dimensions  instead  of  two  can  make  it  easier  for  users  to  understand  the  data.  In  addition,  the  error 
rate  in  identifying  routes  in  3D  graphs  is  much  smaller  than  2D  [Ware'93].  The  CyberNet  system  [Dos 
Santos'00]  shows  that  mapping  large  amount  of  (dynamic)  information  to  3D  representation  is  beneficial, 
regardless  of  the  type  of  metaphors  (real  or  virtual)  used.  Also,  3D  representations  have  been  shown  to 
better  support  spatial  memory  tasks  than  2D  [Tavanti'01].  In  addition,  the  use  of  3D  representations  of 
software  in  new  mediums,  such  as  virtual  reality  environments,  are  starting  to  be  explored  [Knight  99, 
Maletic*01]. 

MacKinlay  [MacKinlay’86]  defined  two  criteria  to  evaluate  the  mapping  of  data  to  a  visual  metaphor: 
expressiveness  and  effectiveness.  These  criteria  were  used  in  2D  mappings,  but  can  also  be  applied  for  3D 
mappings.  Expressiveness  refers  to  the  capability  of  the  metaphor  to  visually  represent  all  the  information 

DISTRIBUTION  STATEMENT  A 

i  Approved  for  Public  Release 
Distribution  Unlimited 


20030103  174 


we  desire  to  visualize.  For  instance,  if  the  number  of  visual  parameters  available  in  the  metaphor  for 
displaying  information  is  fewer  than  the  number  of  data  values  we  wish  to  visualize,  the  metaphor  will  not 
be  able  to  meet  the  expressiveness  criterion.  The  relationship  between  data  values  and  visual  parameters 
has  to  be  a  univocal  relationship;  otherwise,  if  more  than  one  data  value  is  mapped  onto  the  same  visual 
parameter  than  it  will  be  impossible  to  distinguish  one  value’s  influence  from  the  other.  On  the  other  hand, 
there  can  always  be  visual  parameters  that  are  not  used  to  map  information,  as  long  as  there  is  no  need  for 
them  to  be  utilized. 

The  second  criterion,  effectiveness ,  relates  to  the  efficacy  of  the  metaphor  as  a  means  of  representing  the 
information.  Along  the  effectiveness  dimension  we  can  further  distinguish  several  criteria:  effectiveness 
regarding  the  information  passing  as  visually  perceived,  regarding  aesthetic  concerns,  regarding 
optimization  (e.g.,  number  of  polygons  needed  to  render  the  view).  In  the  case  of  quantitative  data  (e.g., 
software  metrics,  LOC,  trace  data),  not  only  the  number  of  visual  parameters  has  to  be  sufficient  to  map  all 
the  data,  but  also,  they  must  be  able  to  map  the  right  data.  There  are  visual  parameters  that  are  not  able  to 
map  a  specific  category  of  data;  for  instance,  shape  is  not  useful  for  mapping  quantitative  data,  while  the 
size  of  a  metaphor  is  adequate.  Effectiveness  implies  the  categorization  of  the  visual  parameters  according 
to  its  capabilities  of  encoding  the  different  types  of  information.  Moreover,  this  also  implies  categorizing 
the  information  according  to  its  importance  so  that  information  that  is  more  important  can  be  encoded  more 
efficiently  when  options  must  be  taken.  This  categorization  of  the  importance  of  the  information  has  two 
expressions:  one  is  an  assigned  importance  of  the  information  in  the  context  of  a  software  system;  the  other 
is  a  preference  of  the  user.  Nonetheless,  the  user  may  choose  to  override  this  and  define  his  own 
importance  of  the  data,  according  to  his  priorities  is  usually  the  first  step  to  understand  a  phenomenon  or 
system.  Although  these  characteristics  of  data  apply  mostly  to  data  visualization,  they  must  be  taken  into 
consideration  when  visualizing  a  software  system. 

The  prototype  we  developed,  IMSOvision,  automatically  translates  given  source  code  and  generates  the 
VRML  source  for  the  visualization.  It  uses  a  visual  display  language  to  incorporate  a  flexible  abstraction 
mechanism.  This  will  allow  for  better  navigation  of  the  very  large  graphs  that  represent  software  system. 
We  are  utilizing  an  abstraction  mechanism  similar  to  what  Colin  Ware  used  to  represent  large-scale  graphs. 
Modules  or  subsystems  are  represented  encased  in  semi-transparent  cubes.  These  can  be  shrunk  and 
enlarged  with  respect  to  the  current  cognitive  importance.  The  contents  of  the  cube  will  always  be  visible 
but  in  the  minimized  mode  only  a  thumb  nail  of  the  internals  is  discernabie.  Additionally,  we  have  added 
the  names  of  classes  and  attributes  to  the  visualization.  This  system  is  working,  however  it  is  a  prototype 
and  much  work  is  necessary  to  have  it  fully  functional. 

We  have  encountered  two  interesting  sub-problems  that  require  further  research.  The  first  is  the 
development  of  a  3D  layout  algorithm  that  readily  support  large  complex  graphs.  A  variety  of  graph  layout 
algorithms  exist,  but  few  that  apply  to  a  3D  space.  Additionally,  the  domain  (in  this  case  software)  may 
allow  the  nodes  to  be  weighted  by  overall  importance.  This  weighting  could  vary  based  on  the  particular 
task.  The  second  problem  is  defining  a  natural  abstraction  mechanism  to  support  better  navigation  of  large 
graphs. 

SIGNIFICANCE:  The  significance  of  this  research  is  two  fold.  Firstly,  it  is  investigating  how  to  visualize 
large  complex  graphs  in  immersive  virtual  reality.  This  new  visual  medium  should  prove  to  be  very 
powerful  for  this  general  application.  Secondly,  the  research  is  addressing  the  complex  domain  of  software 
visualization.  Any  tool  or  technique  that  provides  benefits  to  software  developers  has  the  potential  for  large 
cost  savings  and  quality  improvement.  Practical  software  visualization  must  provide  tools  to  select  and 
display  just  the  information  of  interest.  A  practical  software  visualization  system  can  be  achieved  by 
focusing  on  abstractions.  We  view  the  problem  of  software  visualization  within  these  five  dimensions:  1) 
Target  -  what  to  represent?  2)  Medium  -  where  to  represent?  3)  Representation  -  how  to  represent?  4) 
Audience  -  who  will  use  the  visualization?  5)  Tasks  -  why  is  the  visualization  needed? 

There  is  variety  of  information  with  respect  to  software  that  can  be  represented.  Metric  information,  static 
call  graphs,  runtime  profiles,  and  trace  information  are  all  possible  targets  for  visual  representation.  The 
type  of  information  needed  is  in  direct  relation  to  the  task.  The  type  of  medium  we  display  the  information 
on  can  greatly  vary.  In  our  work,  we  are  investigating  using  mediums  besides  the  21 -inch  monitor.  The 
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need  for  more  screen  restate  is  motivating  much  research  into  new  display  technology.  In  addition,  the  use 
of  multi-type  mediums  (e.g.,  laptop  in  a  VE)  could  be  of  benefit.  In  general,  a  software  visualization  system 
should  determine  the  abstraction  level  of  the  information  it  depicts  about  the  software  system.  It  should  use 
a  visual  language  or  mapping  to  translate  source  code  (and  possibly  external  documentation)  into  a  visual 
representation.  The  choice  of  mapping  depends  on  the  type  of  information  it  represents  and  the  media  used 
in  the  representation.  The  user  tasks  (i.e.,  manipulation,  navigation,  etc.)  that  the  system  supports,  including 
program  comprehension  tasks,  should  be  specified.  In  addition,  the  representation  must  support  multi¬ 
views  and  multi-levels  of  abstractions.  Lastly,  there  are  very  different  types  of  audiences  and  a  wide  range 
of  software  engineering  tasks  that  a  software  visualization  environment  can  be  directed  towards.  Audiences 
may  range  from  teaching  novice  programmers  better  development  techniques  to  expert  developers 
maintaining  a  20  million-line  program.  This  dimension  is  directly  tied  with  the  types  of  tasks,  from 
debugging  and  optimization  to  re-engineering  and  maintenance. 

The  DoD  must  maintain  and  manage  a  very  large  amount  of  software.  Tools  the  support  these  tasks  are 
vital  for  understanding  the  quality  and  capabilities  of  these  software  systems.  This  research  is  directly 
addressing  these  issues.  Additionally,  this  type  of  visualization  framework  may  be  applied  to  other  domains 
for  example,  visualizing  the  relationships  and  interaction  of  (non-software)  systems  or  products  used  by  the 
DoD.  We  believe  this  could  be  used  to  visualize  the  interacting  components  of  complex  systems  such  as  a 
ship. 

PATENT  INFORMATION:  No  new  patents  or  patent  applications  were  generated  as  a  result  of  this  grant. 

AWARD  INFORMATION:  No  major  awards,  however  the  PI  did  leave  the  University  of  Memphis  to 
accept  a  new  position  at  Kent  State  University. 
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